{
 "nbformat": 4,
 "nbformat_minor": 0,
 "metadata": {
  "colab": {
   "name": "Question_Pair_Classification.ipynb",
   "provenance": [],
   "collapsed_sections": [],
   "toc_visible": true
  },
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tOHVDa9DQQR5",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "![JohnSnowLabs](https://nlp.johnsnowlabs.com/assets/images/logo.png)\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/nlu/blob/master/examples/colab/component_examples/classifiers/Question_Pair_Classification.ipynb)\n",
    "\n",
    "\n",
    "# [Question Pair Classifier](https://nlp.johnsnowlabs.com/2021/08/13/classifierdl_electra_questionpair_en.html)\n",
    "\n",
    "\n",
    "Identifies whether two question sentences are semantically repetitive or different.\n",
    "\n",
    "\n",
    "It can be used to classify similar questions therefore automatically showing answers to previously answered questions therefore people will get answers quicker\n",
    "##How to Use\n",
    "\n",
    "* The model is trained with `en.embed_sentence.electra_large_uncased` embeddings therefore the same embeddings should be used in the prediction pipeline.\n",
    "\n",
    "* The question pairs should be identified with “q1” and “q2” in the text. The input text format should be as follows : `text = \"q1: What is your name? q2: Who are you?\"`\n",
    "\n",
    "<br>\n",
    "\n",
    "This model can be used to predict the following news categories \n",
    "`almost_same`, `not_same`\n",
    "\n",
    "<br>\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "The data Source used to train this can be found [here](https://quoradata.quora.com/First-Quora-Dataset-Release-Question-Pairs)\n",
    "\n",
    "<br>\n",
    "\n",
    "##Benchmark on Dataset \n",
    "![image.png]()"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "##1.Setup Java 8 and NLU"
   ],
   "metadata": {
    "id": "HIxATMI7ixJx",
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "SF5-Z-U4jukd",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "0f6b53b9-d963-45fc-aa64-915970a47ade",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "source": [
    "!wget https://raw.githubusercontent.com/JohnSnowLabs/nlu/master/scripts/colab_setup.sh -O - | bash\n",
    "import nlu"
   ],
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "--2022-05-19 21:04:15--  https://raw.githubusercontent.com/JohnSnowLabs/nlu/master/scripts/colab_setup.sh\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 1665 (1.6K) [text/plain]\n",
      "Saving to: ‘STDOUT’\n",
      "\n",
      "\r-                     0%[                    ]       0  --.-KB/s               \r-                   100%[===================>]   1.63K  --.-KB/s    in 0s      \n",
      "\n",
      "2022-05-19 21:04:15 (36.5 MB/s) - written to stdout [1665/1665]\n",
      "\n",
      "Installing  NLU 3.4.4rc1 with  PySpark 3.0.3 and Spark NLP 3.4.3 for Google Colab ...\n",
      "Hit:1 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  InRelease\n",
      "Ign:2 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  InRelease\n",
      "Hit:3 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  Release\n",
      "Hit:4 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran40/ InRelease\n",
      "Hit:5 http://archive.ubuntu.com/ubuntu bionic InRelease\n",
      "Hit:6 http://security.ubuntu.com/ubuntu bionic-security InRelease\n",
      "Hit:7 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic InRelease\n",
      "Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease\n",
      "Hit:10 http://archive.ubuntu.com/ubuntu bionic-backports InRelease\n",
      "Hit:11 http://ppa.launchpad.net/cran/libgit2/ubuntu bionic InRelease\n",
      "Hit:12 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu bionic InRelease\n",
      "Hit:13 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic InRelease\n",
      "Reading package lists... Done\n",
      "tar: spark-3.0.2-bin-hadoop2.7.tgz: Cannot open: No such file or directory\n",
      "tar: Error is not recoverable: exiting now\n",
      "\u001B[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "nlu-tmp 3.4.4rc9 requires spark-nlp<3.5.0,>=3.4.4, but you have spark-nlp 3.4.3 which is incompatible.\u001B[0m\n",
      "Requirement already satisfied: nlu_tmp==3.4.4rc9 in /usr/local/lib/python3.7/dist-packages (3.4.4rc9)\n",
      "Requirement already satisfied: pyarrow>=0.16.0 in /usr/local/lib/python3.7/dist-packages (from nlu_tmp==3.4.4rc9) (6.0.1)\n",
      "Requirement already satisfied: dataclasses in /usr/local/lib/python3.7/dist-packages (from nlu_tmp==3.4.4rc9) (0.6)\n",
      "Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from nlu_tmp==3.4.4rc9) (1.21.6)\n",
      "Requirement already satisfied: pandas>=1.3.5 in /usr/local/lib/python3.7/dist-packages (from nlu_tmp==3.4.4rc9) (1.3.5)\n",
      "Collecting spark-nlp<3.5.0,>=3.4.4\n",
      "  Using cached spark_nlp-3.4.4-py2.py3-none-any.whl (145 kB)\n",
      "Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.3.5->nlu_tmp==3.4.4rc9) (2.8.2)\n",
      "Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.3.5->nlu_tmp==3.4.4rc9) (2022.1)\n",
      "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas>=1.3.5->nlu_tmp==3.4.4rc9) (1.15.0)\n",
      "Installing collected packages: spark-nlp\n",
      "  Attempting uninstall: spark-nlp\n",
      "    Found existing installation: spark-nlp 3.4.3\n",
      "    Uninstalling spark-nlp-3.4.3:\n",
      "      Successfully uninstalled spark-nlp-3.4.3\n",
      "Successfully installed spark-nlp-3.4.4\n"
     ]
    }
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "##2.Load the mdoel and make Sample Predictions "
   ],
   "metadata": {
    "id": "XiVdjjfzij2R",
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "pipeline = nlu.load('en.classify.questionpair')\n",
    "pipeline.predict(\"q1: What is your favorite movie? q2: Which movie do you like most?\",output_level = 'document')"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 298
    },
    "id": "zzWPoMYZsVj2",
    "outputId": "da44c279-aa14-44b4-fbe6-4fccee159945",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "classifierdl_electra_questionpair download started this may take some time.\n",
      "Approximate size to download 22.7 MB\n",
      "[OK!]\n",
      "sent_electra_large_uncased download started this may take some time.\n",
      "Approximate size to download 1.2 GB\n",
      "[OK!]\n"
     ]
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "                                            document questionpair  \\\n",
       "0  q1: What is your favorite movie? q2: Which mov...  almost_same   \n",
       "\n",
       "  questionpair_confidence_confidence  \\\n",
       "0                                1.0   \n",
       "\n",
       "       sentence_embedding_sent_electra_large_uncased  \n",
       "0  [0.1028885468840599, 0.09209786355495453, -0.0...  "
      ],
      "text/html": [
       "\n",
       "  <div id=\"df-09762542-95c0-4ff4-a4a9-56a01d63c545\">\n",
       "    <div class=\"colab-df-container\">\n",
       "      <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>questionpair</th>\n",
       "      <th>questionpair_confidence_confidence</th>\n",
       "      <th>sentence_embedding_sent_electra_large_uncased</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>q1: What is your favorite movie? q2: Which mov...</td>\n",
       "      <td>almost_same</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[0.1028885468840599, 0.09209786355495453, -0.0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-09762542-95c0-4ff4-a4a9-56a01d63c545')\"\n",
       "              title=\"Convert this dataframe to an interactive table.\"\n",
       "              style=\"display:none;\">\n",
       "        \n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "       width=\"24px\">\n",
       "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
       "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
       "  </svg>\n",
       "      </button>\n",
       "      \n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      flex-wrap:wrap;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "      <script>\n",
       "        const buttonEl =\n",
       "          document.querySelector('#df-09762542-95c0-4ff4-a4a9-56a01d63c545 button.colab-df-convert');\n",
       "        buttonEl.style.display =\n",
       "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "        async function convertToInteractive(key) {\n",
       "          const element = document.querySelector('#df-09762542-95c0-4ff4-a4a9-56a01d63c545');\n",
       "          const dataTable =\n",
       "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                     [key], {});\n",
       "          if (!dataTable) return;\n",
       "\n",
       "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "            + ' to learn more about interactive tables.';\n",
       "          element.innerHTML = '';\n",
       "          dataTable['output_type'] = 'display_data';\n",
       "          await google.colab.output.renderOutput(dataTable, element);\n",
       "          const docLink = document.createElement('div');\n",
       "          docLink.innerHTML = docLinkHtml;\n",
       "          element.appendChild(docLink);\n",
       "        }\n",
       "      </script>\n",
       "    </div>\n",
       "  </div>\n",
       "  "
      ]
     },
     "metadata": {},
     "execution_count": 4
    }
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "##3.Define Sample Sentences"
   ],
   "metadata": {
    "id": "_XcMyXVXsdYW",
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "sample_sentences = [\n",
    "\"q1: What is your favorite movie? q2: Which movie do you like most?\",\n",
    "\"q1: What is your favorite movie? q2: Which movie genre would you like to watch?\",\n",
    "\"q1: Are you hungry? q2: You feel like eating?\",\n",
    "\"q1: what do we have for dinner? q2: What are we going to eat tonight?\"\n",
    "]"
   ],
   "metadata": {
    "id": "c_ty0gP6sglE",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "##4.Predict on Sample Sentences"
   ],
   "metadata": {
    "id": "qpD0_HqksiLw",
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "pipeline.predict(sample_sentences,output_level = 'document')"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 479
    },
    "id": "zynQvfNmWI50",
    "outputId": "e856105c-80f4-4944-bdea-34c3e3936667",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "                                            document questionpair  \\\n",
       "0  q1: What is your favorite movie? q2: Which mov...  almost_same   \n",
       "1  q1: What is your favorite movie? q2: Which mov...     not_same   \n",
       "2      q1: Are you hungry? q2: You feel like eating?  almost_same   \n",
       "3  q1: what do we have for dinner? q2: What are w...  almost_same   \n",
       "\n",
       "  questionpair_confidence_confidence  \\\n",
       "0                                1.0   \n",
       "1                           0.690603   \n",
       "2                                1.0   \n",
       "3                           0.909598   \n",
       "\n",
       "       sentence_embedding_sent_electra_large_uncased  \n",
       "0  [[0.05963423475623131, 0.05926075577735901, -0...  \n",
       "1  [[0.08518768846988678, 0.046159692108631134, -...  \n",
       "2  [[-0.023364093154668808, -0.046164620667696, 0...  \n",
       "3  [[0.09144271165132523, 0.1139305904507637, 0.0...  "
      ],
      "text/html": [
       "\n",
       "  <div id=\"df-a85e02e9-4934-4886-ab88-21232f687ec7\">\n",
       "    <div class=\"colab-df-container\">\n",
       "      <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>questionpair</th>\n",
       "      <th>questionpair_confidence_confidence</th>\n",
       "      <th>sentence_embedding_sent_electra_large_uncased</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>q1: What is your favorite movie? q2: Which mov...</td>\n",
       "      <td>almost_same</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[[0.05963423475623131, 0.05926075577735901, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>q1: What is your favorite movie? q2: Which mov...</td>\n",
       "      <td>not_same</td>\n",
       "      <td>0.690603</td>\n",
       "      <td>[[0.08518768846988678, 0.046159692108631134, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>q1: Are you hungry? q2: You feel like eating?</td>\n",
       "      <td>almost_same</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[[-0.023364093154668808, -0.046164620667696, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>q1: what do we have for dinner? q2: What are w...</td>\n",
       "      <td>almost_same</td>\n",
       "      <td>0.909598</td>\n",
       "      <td>[[0.09144271165132523, 0.1139305904507637, 0.0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a85e02e9-4934-4886-ab88-21232f687ec7')\"\n",
       "              title=\"Convert this dataframe to an interactive table.\"\n",
       "              style=\"display:none;\">\n",
       "        \n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "       width=\"24px\">\n",
       "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
       "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
       "  </svg>\n",
       "      </button>\n",
       "      \n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      flex-wrap:wrap;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "      <script>\n",
       "        const buttonEl =\n",
       "          document.querySelector('#df-a85e02e9-4934-4886-ab88-21232f687ec7 button.colab-df-convert');\n",
       "        buttonEl.style.display =\n",
       "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "        async function convertToInteractive(key) {\n",
       "          const element = document.querySelector('#df-a85e02e9-4934-4886-ab88-21232f687ec7');\n",
       "          const dataTable =\n",
       "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                     [key], {});\n",
       "          if (!dataTable) return;\n",
       "\n",
       "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "            + ' to learn more about interactive tables.';\n",
       "          element.innerHTML = '';\n",
       "          dataTable['output_type'] = 'display_data';\n",
       "          await google.colab.output.renderOutput(dataTable, element);\n",
       "          const docLink = document.createElement('div');\n",
       "          docLink.innerHTML = docLinkHtml;\n",
       "          element.appendChild(docLink);\n",
       "        }\n",
       "      </script>\n",
       "    </div>\n",
       "  </div>\n",
       "  "
      ]
     },
     "metadata": {},
     "execution_count": 12
    }
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "##5.Take a look at the parmaters of the pipeline"
   ],
   "metadata": {
    "id": "cEf6CWsxtDWJ",
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "pipeline.print_info()"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3TgLjxNltPCp",
    "outputId": "75c80fcf-605a-4b2e-9401-1e67202136a2",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "The following parameters are configurable for this NLU pipeline (You can copy paste the examples) :\n",
      ">>> component_list['classifier_dl@sent_electra_large_uncased'] has settable params:\n",
      "component_list['classifier_dl@sent_electra_large_uncased'].setClasses(['almost_same', 'not_same'])  | Info: get the tags used to trained this ClassifierDLModel | Currently set to : ['almost_same', 'not_same']\n",
      "component_list['classifier_dl@sent_electra_large_uncased'].setStorageRef('sent_electra_large_uncased')  | Info: unique reference name for identification | Currently set to : sent_electra_large_uncased\n",
      ">>> component_list['bert_sentence_embeddings@sent_electra_large_uncased'] has settable params:\n",
      "component_list['bert_sentence_embeddings@sent_electra_large_uncased'].setBatchSize(8)  | Info: Size of every batch | Currently set to : 8\n",
      "component_list['bert_sentence_embeddings@sent_electra_large_uncased'].setIsLong(False)  | Info: Use Long type instead of Int type for inputs buffer - Some Bert models require Long instead of Int. | Currently set to : False\n",
      "component_list['bert_sentence_embeddings@sent_electra_large_uncased'].setMaxSentenceLength(128)  | Info: Max sentence length to process | Currently set to : 128\n",
      "component_list['bert_sentence_embeddings@sent_electra_large_uncased'].setDimension(1024)  | Info: Number of embedding dimensions | Currently set to : 1024\n",
      "component_list['bert_sentence_embeddings@sent_electra_large_uncased'].setCaseSensitive(False)  | Info: whether to ignore case in tokens for embeddings matching | Currently set to : False\n",
      "component_list['bert_sentence_embeddings@sent_electra_large_uncased'].setStorageRef('sent_electra_large_uncased')  | Info: unique reference name for identification | Currently set to : sent_electra_large_uncased\n",
      ">>> component_list['document_assembler'] has settable params:\n",
      "component_list['document_assembler'].setCleanupMode('shrink')  | Info: possible values: disabled, inplace, inplace_full, shrink, shrink_full, each, each_full, delete_full | Currently set to : shrink\n"
     ]
    }
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "Looking Good! Let's test this model on a labelled dataset to see how it performs "
   ],
   "metadata": {
    "id": "1hF8dmLIkbSz",
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "##6.Download Data\n",
    "\n",
    "we are going to test the model on [this](https://www.kaggle.com/datasets/quora/question-pairs-dataset) dataset \n",
    "\n",
    "Quora's first public dataset is related to the problem of identifying duplicate questions. At Quora, an important product principle is that there should be a single question page for each logically distinct question. For example, the queries “What is the most populous state in the USA?” and “Which state in the United States has the most people?” should not exist separately on Quora because the intent behind both is identical. Having a canonical page for each logically distinct query makes knowledge-sharing more efficient in many ways: for example, knowledge seekers can access all the answers to a question in a single location, and writers can reach a larger readership than if that audience was divided amongst several pages.\n",
    "\n",
    "The dataset is based on actual data from Quora and will give anyone the opportunity to train and test models of semantic equivalence.\n"
   ],
   "metadata": {
    "id": "uRbusvKOkVac",
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "!wget http://ckl-it.de/wp-content/uploads/2022/05/questions_pair_data-1.csv"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "KvD0rZTBrnrL",
    "outputId": "2384dcd0-ef12-4021-bb78-2283347fea0f",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "--2022-05-19 21:33:53--  http://ckl-it.de/wp-content/uploads/2022/05/questions_pair_data-1.csv\n",
      "Resolving ckl-it.de (ckl-it.de)... 217.160.0.108, 2001:8d8:100f:f000::209\n",
      "Connecting to ckl-it.de (ckl-it.de)|217.160.0.108|:80... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 57120912 (54M) [text/csv]\n",
      "Saving to: ‘questions_pair_data-1.csv’\n",
      "\n",
      "questions_pair_data 100%[===================>]  54.47M  17.5MB/s    in 3.9s    \n",
      "\n",
      "2022-05-19 21:33:58 (14.1 MB/s) - ‘questions_pair_data-1.csv’ saved [57120912/57120912]\n",
      "\n"
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "import pandas  as pd \n",
    "df = pd.read_csv(\"questions_pair_data-1.csv\")\n",
    "df"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 424
    },
    "id": "n-6bOT4krpr8",
    "outputId": "3db99169-938c-4183-cd39-614924709134",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "                                                     text            y\n",
       "0       q1: What is the step by step guide to invest i...     not_same\n",
       "1       q1: What is the story of Kohinoor (Koh-i-Noor)...     not_same\n",
       "2       q1: How can I increase the speed of my interne...     not_same\n",
       "3       q1: Why am I mentally very lonely? How can I s...     not_same\n",
       "4       q1: Which one dissolve in water quikly sugar, ...     not_same\n",
       "...                                                   ...          ...\n",
       "404346  q1: How many keywords are there in the Racket ...     not_same\n",
       "404347  q1: Do you believe there is life after death? ...  almost_same\n",
       "404348        q1: What is one coin? q2: What's this coin?     not_same\n",
       "404349  q1: What is the approx annual cost of living w...     not_same\n",
       "404350  q1: What is like to have sex with cousin? q2: ...     not_same\n",
       "\n",
       "[404351 rows x 2 columns]"
      ],
      "text/html": [
       "\n",
       "  <div id=\"df-7b05e384-ec3e-42f7-90ea-bfb2dd2d9a67\">\n",
       "    <div class=\"colab-df-container\">\n",
       "      <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>q1: What is the step by step guide to invest i...</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>q1: What is the story of Kohinoor (Koh-i-Noor)...</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>q1: How can I increase the speed of my interne...</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>q1: Why am I mentally very lonely? How can I s...</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>q1: Which one dissolve in water quikly sugar, ...</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>404346</th>\n",
       "      <td>q1: How many keywords are there in the Racket ...</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>404347</th>\n",
       "      <td>q1: Do you believe there is life after death? ...</td>\n",
       "      <td>almost_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>404348</th>\n",
       "      <td>q1: What is one coin? q2: What's this coin?</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>404349</th>\n",
       "      <td>q1: What is the approx annual cost of living w...</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>404350</th>\n",
       "      <td>q1: What is like to have sex with cousin? q2: ...</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>404351 rows × 2 columns</p>\n",
       "</div>\n",
       "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-7b05e384-ec3e-42f7-90ea-bfb2dd2d9a67')\"\n",
       "              title=\"Convert this dataframe to an interactive table.\"\n",
       "              style=\"display:none;\">\n",
       "        \n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "       width=\"24px\">\n",
       "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
       "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
       "  </svg>\n",
       "      </button>\n",
       "      \n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      flex-wrap:wrap;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "      <script>\n",
       "        const buttonEl =\n",
       "          document.querySelector('#df-7b05e384-ec3e-42f7-90ea-bfb2dd2d9a67 button.colab-df-convert');\n",
       "        buttonEl.style.display =\n",
       "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "        async function convertToInteractive(key) {\n",
       "          const element = document.querySelector('#df-7b05e384-ec3e-42f7-90ea-bfb2dd2d9a67');\n",
       "          const dataTable =\n",
       "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                     [key], {});\n",
       "          if (!dataTable) return;\n",
       "\n",
       "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "            + ' to learn more about interactive tables.';\n",
       "          element.innerHTML = '';\n",
       "          dataTable['output_type'] = 'display_data';\n",
       "          await google.colab.output.renderOutput(dataTable, element);\n",
       "          const docLink = document.createElement('div');\n",
       "          docLink.innerHTML = docLinkHtml;\n",
       "          element.appendChild(docLink);\n",
       "        }\n",
       "      </script>\n",
       "    </div>\n",
       "  </div>\n",
       "  "
      ]
     },
     "metadata": {},
     "execution_count": 23
    }
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "Let's take  a Peek at the distribution of the labels "
   ],
   "metadata": {
    "id": "mUl3_RmhuSHW",
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "df.y.value_counts().plot.bar(title='Distribution of Labels')"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 354
    },
    "id": "n_c6k0EaNpM9",
    "outputId": "a32c041a-8f49-44e2-8f2b-513c4975ca4b",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f735b68ce10>"
      ]
     },
     "metadata": {},
     "execution_count": 24
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ],
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAFACAYAAABJOoDBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAar0lEQVR4nO3dfbildV3v8fdHQEGRJ5kIARnUOdbIKdQJKTtHjYJB7IBFBpWMSpKKHS0r8aFA1NLOEa+jx+hQTIClSJoxJogTodYxlNGIB5WYgyADCCODgArIw/f8sX5b1izWb/Y87rVnz/t1Xevaa33v+/7d3zVs1mffv/tea6WqkCRpnMdMugFJ0uxlSEiSugwJSVKXISFJ6jIkJEldhoQkqcuQ0KyT5M+T/OFmGuspSb6bZLv2+LNJfnNzjN3GuyjJks013gbs951Jvp3kW5txzPlJKsn2M7mtZjdDQjMqyQ1J7k1yT5LvJPlCklcn+eHvYlW9uqresZ5j/fy61qmqb1bVzlX10Gbo/dQkfz0y/hFVdc6mjr2BfTwFeCOwsKp+dMzyFyRZNZM9ae4yJDQJv1hVTwT2B94NvAk4a3PvZA7/VfsU4I6qun3SjWjuMyQ0MVV1V1UtA34VWJLkQIAkZyd5Z7u/Z5J/aEcda5L8c5LHJPkQgxfLT7bppD8YmvI4Ick3gX/qTIM8LcmXktyd5IIke7R9Peov8KmjlSSLgbcAv9r29+9t+Q+nr1pfb0tyY5Lbk5ybZNe2bKqPJUm+2aaK3tr7t0mya9t+dRvvbW38nweWA09ufZy9If/mSY5M8m/tud+U5NQxq70yyS1Jbk3ye0PbPibJyUn+X5I7kpw/9W83Zj8vT3J9O2L8RpJf35A+NXsYEpq4qvoSsAr4L2MWv7EtmwfsxeCFuqrqZcA3GRyV7FxVfzq0zfOBHwcO7+zyeOCVwN7Ag8D716PHTwN/DHy07e8nx6z28nZ7IfBUYGfgf4+s87PAM4BDgT9K8uOdXX4A2LWN8/zW8yuq6h+BI4BbWh8vn673Ed9rY+0GHAm8JsnRI+u8EFgAHAa8aWhK77eBo1s/TwbuBD44uoMkT2Dwb3pEO2L8GeCKDexTs4QhodniFmDcX6UPMHgx37+qHqiqf67pP3Ds1Kr6XlXd21n+oaq6uqq+B/wh8NKpE9ub6NeB06vq+qr6LvBm4NiRo5i3V9W9VfXvwL8Djwqb1suxwJur6p6qugF4L/CyTW2wqj5bVVdV1cNVdSXwEQYv+sPe3v79rgL+Cjiu1V8NvLWqVlXV/cCpwDGdab2HgQOT7FRVt1bVNZvauybDkNBssQ+wZkz9fwArgc+06YuT12OsmzZg+Y3ADsCe69Xluj25jTc89vYMjoCmDF+N9H0GRxuj9mw9jY61z6Y2mOS5SS5t01h3MXjhH33uo/8+T2739wc+0ab+vgN8DXiItZ8fLXx/tY19a5JPJfmxTe1dk2FIaOKS/BSDF8B/GV3W/pJ+Y1U9FfhvwO8mOXRqcWfI6Y409hu6/xQGRyvfZjAV8/ihvrZjMM21vuPewuCFdHjsB4Hbptlu1LdbT6Nj3byB44zzYWAZsF9V7Qr8OZCRdUb/fW5p929iMIW029Btx6p6VF9VdXFV/QKDo8CvA3+xGXrXBBgSmpgkuyR5MXAe8NdtemN0nRcneXqSAHcx+Mv14bb4NgZz9hvqN5IsTPJ44DTgY+0S2f8Admwnd3cA3gY8bmi724D5w5frjvgI8DtJDkiyM4+cw3hwQ5prvZwPvCvJE5PsD/wu8Nfr3nJtSXYcuQV4IrCmqu5LcjDwa2M2/cMkj0/yTOAVwEdb/c9bT/u38eclOWrMfvdKclQ7N3E/8F0e+W+mrYwhoUn4ZJJ7GPxl+lbgdAYvRuMsAP6RwQvNvwJ/VlWXtmV/ArytTX/8Xmf7cT4EnM1g6mdH4L/D4Gor4LXAXzL4q/17DE6aT/nb9vOOJF8ZM+7SNvbngW8A9zE42bsxfrvt/3oGR1gfbuOvr32Ae0duT2Pw/E5r//5/xCCMRn2OwRTfJcD/rKrPtPr/YnAU8pm2/WXAc8ds/xgGoXYLgynE5wOv2YDeNYvELx2SJPV4JCFJ6jIkJEldhoQkqcuQkCR1zbkPQNtzzz1r/vz5k25DkrYqX/7yl79dVfNG63MuJObPn8+KFSsm3YYkbVWS3Diu7nSTJKnLkJAkdRkSkqQuQ0KS1GVISJK6DAlJUpchIUnqMiQkSV2GhCSpa86943prMf/kT026hTnlhncfOekWpDlp2iOJJPu1L07/apJrkry+1U9NcnOSK9rtRUPbvDnJyiTXJjl8qL641VYOf6F9+7rHL7b6R5M8ttUf1x6vbMvnb84nL0lat/WZbnoQeGNVLQQOAU5KsrAte19VHdRuFwK0ZccCzwQWA3+WZLv2pfIfBI4AFgLHDY3znjbW04E7gRNa/QTgzlZ/X1tPkjRDpg2Jqrq1qr7S7t8DfI3B9+f2HAWcV1X3V9U3GHxX7sHttrKqrq+qHwDnAUe1L2f/OeBjbftzgKOHxjqn3f8YcGhbX5I0AzboxHWb7nkW8MVWel2SK5MsTbJ7q+3D4Avup6xqtV79ScB3qurBkfpaY7Xld7X1R/s6McmKJCtWr169IU9JkrQO6x0SSXYGPg68oaruBs4AngYcBNwKvHeLdLgequrMqlpUVYvmzXvUx6FLkjbSeoVEkh0YBMTfVNXfAVTVbVX1UFU9DPwFg+kkgJuB/YY237fVevU7gN2SbD9SX2ustnzXtr4kaQasz9VNAc4CvlZVpw/V9x5a7SXA1e3+MuDYdmXSAcAC4EvA5cCCdiXTYxmc3F5WVQVcChzTtl8CXDA01pJ2/xjgn9r6kqQZsD7vk3ge8DLgqiRXtNpbGFyddBBQwA3AbwFU1TVJzge+yuDKqJOq6iGAJK8DLga2A5ZW1TVtvDcB5yV5J/BvDEKJ9vNDSVYCaxgEiyRphkwbElX1L8C4K4ouXMc27wLeNaZ+4bjtqup6HpmuGq7fB/zKdD1KkrYMP5ZDktRlSEiSugwJSVKXISFJ6jIkJEldhoQkqcuQkCR1GRKSpC5DQpLUZUhIkroMCUlSlyEhSeoyJCRJXYaEJKnLkJAkdRkSkqQuQ0KS1GVISJK6DAlJUpchIUnqMiQkSV2GhCSpy5CQJHUZEpKkLkNCktRlSEiSugwJSVKXISFJ6jIkJEldhoQkqcuQkCR1TRsSSfZLcmmSrya5JsnrW32PJMuTXNd+7t7qSfL+JCuTXJnk2UNjLWnrX5dkyVD9OUmuatu8P0nWtQ9J0sxYnyOJB4E3VtVC4BDgpCQLgZOBS6pqAXBJewxwBLCg3U4EzoDBCz5wCvBc4GDglKEX/TOAVw1tt7jVe/uQJM2AaUOiqm6tqq+0+/cAXwP2AY4CzmmrnQMc3e4fBZxbA5cBuyXZGzgcWF5Va6rqTmA5sLgt26WqLquqAs4dGWvcPiRJM2CDzkkkmQ88C/gisFdV3doWfQvYq93fB7hpaLNVrbau+qoxddaxD0nSDFjvkEiyM/Bx4A1VdffwsnYEUJu5t7Wsax9JTkyyIsmK1atXb8k2JGmbsl4hkWQHBgHxN1X1d618W5sqov28vdVvBvYb2nzfVltXfd8x9XXtYy1VdWZVLaqqRfPmzVufpyRJWg/rc3VTgLOAr1XV6UOLlgFTVygtAS4Yqh/frnI6BLirTRldDByWZPd2wvow4OK27O4kh7R9HT8y1rh9SJJmwPbrsc7zgJcBVyW5otXeArwbOD/JCcCNwEvbsguBFwErge8DrwCoqjVJ3gFc3tY7rarWtPuvBc4GdgIuajfWsQ9J0gyYNiSq6l+AdBYfOmb9Ak7qjLUUWDqmvgI4cEz9jnH7kCTNDN9xLUnqMiQkSV2GhCSpy5CQJHUZEpKkLkNCktRlSEiSugwJSVKXISFJ6jIkJEldhoQkqcuQkCR1GRKSpC5DQpLUZUhIkroMCUlSlyEhSeoyJCRJXYaEJKnLkJAkdRkSkqQuQ0KS1GVISJK6DAlJUpchIUnqMiQkSV2GhCSpy5CQJHUZEpKkLkNCktRlSEiSugwJSVLXtCGRZGmS25NcPVQ7NcnNSa5otxcNLXtzkpVJrk1y+FB9cautTHLyUP2AJF9s9Y8meWyrP649XtmWz99cT1qStH7W50jibGDxmPr7quqgdrsQIMlC4FjgmW2bP0uyXZLtgA8CRwALgePaugDvaWM9HbgTOKHVTwDubPX3tfUkSTNo2pCoqs8Da9ZzvKOA86rq/qr6BrASOLjdVlbV9VX1A+A84KgkAX4O+Fjb/hzg6KGxzmn3PwYc2taXJM2QTTkn8bokV7bpqN1bbR/gpqF1VrVar/4k4DtV9eBIfa2x2vK72vqPkuTEJCuSrFi9evUmPCVJ0rDtN3K7M4B3ANV+vhd45eZqakNV1ZnAmQCLFi2qSfUhzQXzT/7UpFuYU25495GTbmGTbNSRRFXdVlUPVdXDwF8wmE4CuBnYb2jVfVutV78D2C3J9iP1tcZqy3dt60uSZshGhUSSvYcevgSYuvJpGXBsuzLpAGAB8CXgcmBBu5LpsQxObi+rqgIuBY5p2y8BLhgaa0m7fwzwT219SdIMmXa6KclHgBcAeyZZBZwCvCDJQQymm24Afgugqq5Jcj7wVeBB4KSqeqiN8zrgYmA7YGlVXdN28SbgvCTvBP4NOKvVzwI+lGQlgxPnx27ys5UkbZBpQ6KqjhtTPmtMbWr9dwHvGlO/ELhwTP16HpmuGq7fB/zKdP1JkrYc33EtSeoyJCRJXYaEJKnLkJAkdRkSkqQuQ0KS1GVISJK6DAlJUpchIUnqMiQkSV2GhCSpy5CQJHUZEpKkLkNCktRlSEiSugwJSVKXISFJ6jIkJEldhoQkqcuQkCR1GRKSpC5DQpLUZUhIkroMCUlSlyEhSeoyJCRJXYaEJKnLkJAkdRkSkqQuQ0KS1GVISJK6pg2JJEuT3J7k6qHaHkmWJ7mu/dy91ZPk/UlWJrkyybOHtlnS1r8uyZKh+nOSXNW2eX+SrGsfkqSZsz5HEmcDi0dqJwOXVNUC4JL2GOAIYEG7nQicAYMXfOAU4LnAwcApQy/6ZwCvGtpu8TT7kCTNkGlDoqo+D6wZKR8FnNPunwMcPVQ/twYuA3ZLsjdwOLC8qtZU1Z3AcmBxW7ZLVV1WVQWcOzLWuH1IkmbIxp6T2Kuqbm33vwXs1e7vA9w0tN6qVltXfdWY+rr2IUmaIZt84rodAdRm6GWj95HkxCQrkqxYvXr1lmxFkrYpGxsSt7WpItrP21v9ZmC/ofX2bbV11fcdU1/XPh6lqs6sqkVVtWjevHkb+ZQkSaM2NiSWAVNXKC0BLhiqH9+ucjoEuKtNGV0MHJZk93bC+jDg4rbs7iSHtKuajh8Za9w+JEkzZPvpVkjyEeAFwJ5JVjG4SundwPlJTgBuBF7aVr8QeBGwEvg+8AqAqlqT5B3A5W2906pq6mT4axlcQbUTcFG7sY59SJJmyLQhUVXHdRYdOmbdAk7qjLMUWDqmvgI4cEz9jnH7kCTNHN9xLUnqMiQkSV2GhCSpy5CQJHUZEpKkLkNCktRlSEiSugwJSVKXISFJ6jIkJEldhoQkqcuQkCR1GRKSpC5DQpLUZUhIkroMCUlSlyEhSeoyJCRJXYaEJKnLkJAkdRkSkqQuQ0KS1GVISJK6DAlJUpchIUnqMiQkSV2GhCSpy5CQJHUZEpKkLkNCktRlSEiSugwJSVLXJoVEkhuSXJXkiiQrWm2PJMuTXNd+7t7qSfL+JCuTXJnk2UPjLGnrX5dkyVD9OW38lW3bbEq/kqQNszmOJF5YVQdV1aL2+GTgkqpaAFzSHgMcASxotxOBM2AQKsApwHOBg4FTpoKlrfOqoe0Wb4Z+JUnraUtMNx0FnNPunwMcPVQ/twYuA3ZLsjdwOLC8qtZU1Z3AcmBxW7ZLVV1WVQWcOzSWJGkGbGpIFPCZJF9OcmKr7VVVt7b73wL2avf3AW4a2nZVq62rvmpM/VGSnJhkRZIVq1ev3pTnI0kasv0mbv+zVXVzkh8Blif5+vDCqqoktYn7mFZVnQmcCbBo0aItvj9J2lZs0pFEVd3cft4OfILBOYXb2lQR7eftbfWbgf2GNt+31dZV33dMXZI0QzY6JJI8IckTp+4DhwFXA8uAqSuUlgAXtPvLgOPbVU6HAHe1aamLgcOS7N5OWB8GXNyW3Z3kkHZV0/FDY0mSZsCmTDftBXyiXZW6PfDhqvp0ksuB85OcANwIvLStfyHwImAl8H3gFQBVtSbJO4DL23qnVdWadv+1wNnATsBF7SZJmiEbHRJVdT3wk2PqdwCHjqkXcFJnrKXA0jH1FcCBG9ujJGnT+I5rSVKXISFJ6jIkJEldhoQkqcuQkCR1GRKSpC5DQpLUZUhIkroMCUlSlyEhSeoyJCRJXYaEJKnLkJAkdRkSkqQuQ0KS1GVISJK6DAlJUpchIUnqMiQkSV2GhCSpy5CQJHUZEpKkLkNCktRlSEiSugwJSVKXISFJ6jIkJEldhoQkqcuQkCR1GRKSpC5DQpLUZUhIkrpmfUgkWZzk2iQrk5w86X4kaVsyq0MiyXbAB4EjgIXAcUkWTrYrSdp2zOqQAA4GVlbV9VX1A+A84KgJ9yRJ24ztJ93ANPYBbhp6vAp47uhKSU4ETmwPv5vk2hnobVuxJ/DtSTcxnbxn0h1oAvzd3Lz2H1ec7SGxXqrqTODMSfcxFyVZUVWLJt2HNMrfzZkx26ebbgb2G3q8b6tJkmbAbA+Jy4EFSQ5I8ljgWGDZhHuSpG3GrJ5uqqoHk7wOuBjYDlhaVddMuK1tjdN4mq383ZwBqapJ9yBJmqVm+3STJGmCDAlJUpchIUnqMiQkSV2GhNaSZK8kZyW5qD1emOSESfclTUmyf5Kfb/d3SvLESfc0lxkSGnU2g0uOn9we/wfwhol1Iw1J8irgY8D/aaV9gb+fXEdznyGhUXtW1fnAwzB4rwrw0GRbkn7oJOB5wN0AVXUd8CMT7WiOMyQ06ntJngQUQJJDgLsm25L0Q/e3T4QGIMn2tN9VbRmz+h3XmojfZfDRJ09L8n+BecAxk21J+qHPJXkLsFOSXwBeC3xywj3Nab7jWo/S/jp7BhDg2qp6YMItSQAkeQxwAnAYg9/Pi4G/LF/IthhDQmtp3wZ4JDCfoSPNqjp9Uj1JmhynmzTqk8B9wFW0k9fSbJHkxcA7GHxBzvYMjiaqqnaZaGNzmEcSWkuSK6vqJybdhzROkpXALwFXOcU0M7y6SaMuSnLYpJuQOm4CrjYgZo7TTRp1GfCJdoLwATyc1+zyB8CFST4H3D9V9JzZlmNIaNTpwE/j4bxmp3cB3wV2BB474V62CYaERnk4r9nsyVV14KSb2JYYEhp1PfDZ9gF/Hs5rtrkwyWFV9ZlJN7Kt8OomrSXJKePqVfX2me5FGpXkHuAJDP6A8ZzZDDAkJEldTjdpLUnmMbiC5JkMTg4CUFU/N7GmpCFJdgcWsPbv5+cn19Hc5vskNOpvgK8DBwBvB24ALp9kQ9KUJL8JfJ7BZza9vf08dZI9zXWGhEY9qarOAh6oqs9V1SsBjyI0W7we+Cngxqp6IfAs4DuTbWluc7pJo6Y+8fXWJEcCtwB7TLAfadh9VXVfEpI8rqq+nuQZk25qLjMkNOqdSXYF3gh8ANgF+J3JtiT90KokuzH4ytLlSe4EbpxwT3OaVzdJ2ioleT6wK/Dp4W+r0+blOQmtJcmfJtklyQ5JLkmyOslvTLovCSDJ05I8buohg+89efzkOpr7DAmNOqyq7gZezODKpqcDvz/RjqRHfBx4KMnTgTOB/YAPT7aluc2Q0Kip81RHAn9bVXdNshlpxMNV9SDwEuADVfX7wN4T7mlO88S1Rv1Dkq8D9wKvaW+uu2/CPUlTHkhyHLAE+MVW22GC/cx5nrjWoyTZA7irqh5K8gTgiVX1rbbsF6pq+WQ71LYqyULg1cC/VtVHkhwAvLSq3jPh1uYsQ0IbJMlXqurZk+5DGifJx6vqlyfdx1ziOQltqEy6AWkdnjrpBuYaQ0IbykNPzWb+fm5mhoQkqcuQ0FqG3qjUq90wc91IG8zp0M3MkNCof11Xrap+aQZ7kdaS5PXT1N40g+1sEwwJAZDkR5M8B9gpybOSPLvdXoAfe6DZY8mY2sun7vjd15ufb6bTlMMZ/M+2L3D6UP0e4C2TaEia0t5A92vAAUmWDS3aBVgzma62Db5PQmtJ8stV9fFJ9yENS7I/g29L/BPg5KFF9wBXto/q0BZgSGgt7bP6/wj4r630OeA0P8NJs0H7BIB7q+rhJP8J+DHgoqp6YJpNtZE8J6FRZzH46+yl7XY38FcT7Uh6xOeBHZPsA3wGeBlw9kQ7muM8ktBaklxRVQdNV5MmYepjYZL8NrBTVf2pv59blkcSGnVvkp+depDkeQw+EVaaDZLkp4FfBz7VattNsJ85z6ubNOo1wDnte64B7mT8ZYfSJLwBeDPwiaq6JslTgUsn3NOc5nST1tLeXX0M8DRgN+AuoKrqtIk2Jg1JsjNAVX130r3MdR5JaNQFwHeArwA3T7gXaS1J/jNwLrDH4GFWA8dX1TWT7Wzu8khCa0lydVUdOOk+pHGSfAF4a1Vd2h6/APjjqvqZiTY2h3niWqO+0P5ak2ajJ0wFBEBVfRZ4wuTamfs8ktBaknwVeDrwDeB+Bp+qWVX1ExNtTAKSfILBVOiHWuk3gOdU1Usm19XcZkhoLe3jDx6lqm6c6V6kUUl2B94OTF2m/c/AqVV15+S6mtsMCUlSl1c3SdpqJFnE4FOJ5zP0+uV06JbjkYSkrUaSa4HfB64CHp6qOx265XgkIWlrsrqqlk2/mjYXjyQkbTWSHAocB1zC4Oo7AKrq7ybW1BznkYSkrckrGHyHxA48Mt1UgCGxhXgkIWmrkeTaqnrGpPvYlviOa0lbky8kWTjpJrYlHklI2mok+RqDTyj2EwFmiCEhaavhJwLMPE9cS5r1kuzR7t4z0Ua2QR5JSJr1knyDwVVMaaWpF66p6aanTqSxbYBHEpJmvao6YOp+O6pYAOw4uY62HYaEpK1Gkt8EXg/sC1wBHAJ8ATh0kn3NZV4CK2lr8nrgp4Abq+qFwLMYfA+7thBDQtLW5L6qug8gyeOq6uuAb67bgpxukrQ1WZVkN+DvgeVJ7gS8/HUL8uomSVulJM8HdgU+XVU/mHQ/c5UhIUnq8pyEJKnLkJAkdRkSkqQuQ0KS1PX/ATVkEqCO1/erAAAAAElFTkSuQmCC\n"
     },
     "metadata": {
      "needs_background": "light"
     }
    }
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "##7.Make Predictions with the model"
   ],
   "metadata": {
    "id": "KHxOy5o9uyTG",
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# The dataset is very large so we only take the first 15000 rows\n",
    "df = df.iloc[:15000]"
   ],
   "metadata": {
    "id": "SnWo1ID5Pji2",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "predctions = pipeline.predict(df,output_level = 'document')"
   ],
   "metadata": {
    "id": "QpmPv2q5uVyd",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "predctions = predctions.dropna(subset = ['questionpair'])"
   ],
   "metadata": {
    "id": "rvyeXaHfL9PP",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "predctions"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "j4JpnxjzMOsV",
    "outputId": "b1905216-1b0d-44d9-cced-e27872a2c467",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "                                                document questionpair  \\\n",
       "0      q1: What is the step by step guide to invest i...     not_same   \n",
       "1      q1: What is the story of Kohinoor (Koh-i-Noor)...     not_same   \n",
       "2      q1: How can I increase the speed of my interne...  almost_same   \n",
       "3      q1: Why am I mentally very lonely? How can I s...     not_same   \n",
       "4      q1: Which one dissolve in water quikly sugar, ...     not_same   \n",
       "...                                                  ...          ...   \n",
       "14995  q1: What strikes first time visitors as specia...     not_same   \n",
       "14996  q1: I want to be a poet what should I do to ge...  almost_same   \n",
       "14997  q1: What is borderline personality disorder? q...     not_same   \n",
       "14998  q1: Are there more colors in a rainbow than th...     not_same   \n",
       "14999  q1: I am constantly thinking about something o...     not_same   \n",
       "\n",
       "      questionpair_confidence_confidence  \\\n",
       "0                                    1.0   \n",
       "1                               0.999998   \n",
       "2                                    1.0   \n",
       "3                                    1.0   \n",
       "4                                    1.0   \n",
       "...                                  ...   \n",
       "14995                                1.0   \n",
       "14996                           0.999999   \n",
       "14997                                1.0   \n",
       "14998                                1.0   \n",
       "14999                                1.0   \n",
       "\n",
       "           sentence_embedding_sent_electra_large_uncased  \\\n",
       "0      [[-0.14823496341705322, -0.20211371779441833, ...   \n",
       "1      [[0.10479416698217392, -0.07261374592781067, 0...   \n",
       "2      [[-0.07979179173707962, 0.046571265906095505, ...   \n",
       "3      [[-0.0022265498992055655, -0.04372467845678329...   \n",
       "4      [[-0.08860134333372116, 0.0703229010105133, 0....   \n",
       "...                                                  ...   \n",
       "14995  [[0.05599864944815636, 0.15240998566150665, 0....   \n",
       "14996  [[0.07333197444677353, -0.031826701015233994, ...   \n",
       "14997  [[-0.10858175903558731, -0.12562920153141022, ...   \n",
       "14998  [[0.011639977805316448, -0.07442142814397812, ...   \n",
       "14999  [[0.07383149117231369, -0.001604342251084745, ...   \n",
       "\n",
       "                                                    text            y  \n",
       "0      q1: What is the step by step guide to invest i...     not_same  \n",
       "1      q1: What is the story of Kohinoor (Koh-i-Noor)...     not_same  \n",
       "2      q1: How can I increase the speed of my interne...     not_same  \n",
       "3      q1: Why am I mentally very lonely? How can I s...     not_same  \n",
       "4      q1: Which one dissolve in water quikly sugar, ...     not_same  \n",
       "...                                                  ...          ...  \n",
       "14995  q1: What strikes first time visitors as specia...     not_same  \n",
       "14996  q1: I want to be a poet what should I do to ge...  almost_same  \n",
       "14997  q1: What is borderline personality disorder? q...  almost_same  \n",
       "14998  q1: Are there more colors in a rainbow than th...     not_same  \n",
       "14999  q1: I am constantly thinking about something o...  almost_same  \n",
       "\n",
       "[15000 rows x 6 columns]"
      ],
      "text/html": [
       "\n",
       "  <div id=\"df-4cb56634-426a-4b4c-b762-54da095d3924\">\n",
       "    <div class=\"colab-df-container\">\n",
       "      <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>questionpair</th>\n",
       "      <th>questionpair_confidence_confidence</th>\n",
       "      <th>sentence_embedding_sent_electra_large_uncased</th>\n",
       "      <th>text</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>q1: What is the step by step guide to invest i...</td>\n",
       "      <td>not_same</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[[-0.14823496341705322, -0.20211371779441833, ...</td>\n",
       "      <td>q1: What is the step by step guide to invest i...</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>q1: What is the story of Kohinoor (Koh-i-Noor)...</td>\n",
       "      <td>not_same</td>\n",
       "      <td>0.999998</td>\n",
       "      <td>[[0.10479416698217392, -0.07261374592781067, 0...</td>\n",
       "      <td>q1: What is the story of Kohinoor (Koh-i-Noor)...</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>q1: How can I increase the speed of my interne...</td>\n",
       "      <td>almost_same</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[[-0.07979179173707962, 0.046571265906095505, ...</td>\n",
       "      <td>q1: How can I increase the speed of my interne...</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>q1: Why am I mentally very lonely? How can I s...</td>\n",
       "      <td>not_same</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[[-0.0022265498992055655, -0.04372467845678329...</td>\n",
       "      <td>q1: Why am I mentally very lonely? How can I s...</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>q1: Which one dissolve in water quikly sugar, ...</td>\n",
       "      <td>not_same</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[[-0.08860134333372116, 0.0703229010105133, 0....</td>\n",
       "      <td>q1: Which one dissolve in water quikly sugar, ...</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14995</th>\n",
       "      <td>q1: What strikes first time visitors as specia...</td>\n",
       "      <td>not_same</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[[0.05599864944815636, 0.15240998566150665, 0....</td>\n",
       "      <td>q1: What strikes first time visitors as specia...</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14996</th>\n",
       "      <td>q1: I want to be a poet what should I do to ge...</td>\n",
       "      <td>almost_same</td>\n",
       "      <td>0.999999</td>\n",
       "      <td>[[0.07333197444677353, -0.031826701015233994, ...</td>\n",
       "      <td>q1: I want to be a poet what should I do to ge...</td>\n",
       "      <td>almost_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14997</th>\n",
       "      <td>q1: What is borderline personality disorder? q...</td>\n",
       "      <td>not_same</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[[-0.10858175903558731, -0.12562920153141022, ...</td>\n",
       "      <td>q1: What is borderline personality disorder? q...</td>\n",
       "      <td>almost_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14998</th>\n",
       "      <td>q1: Are there more colors in a rainbow than th...</td>\n",
       "      <td>not_same</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[[0.011639977805316448, -0.07442142814397812, ...</td>\n",
       "      <td>q1: Are there more colors in a rainbow than th...</td>\n",
       "      <td>not_same</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14999</th>\n",
       "      <td>q1: I am constantly thinking about something o...</td>\n",
       "      <td>not_same</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[[0.07383149117231369, -0.001604342251084745, ...</td>\n",
       "      <td>q1: I am constantly thinking about something o...</td>\n",
       "      <td>almost_same</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>15000 rows × 6 columns</p>\n",
       "</div>\n",
       "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4cb56634-426a-4b4c-b762-54da095d3924')\"\n",
       "              title=\"Convert this dataframe to an interactive table.\"\n",
       "              style=\"display:none;\">\n",
       "        \n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "       width=\"24px\">\n",
       "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
       "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
       "  </svg>\n",
       "      </button>\n",
       "      \n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      flex-wrap:wrap;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "      <script>\n",
       "        const buttonEl =\n",
       "          document.querySelector('#df-4cb56634-426a-4b4c-b762-54da095d3924 button.colab-df-convert');\n",
       "        buttonEl.style.display =\n",
       "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "        async function convertToInteractive(key) {\n",
       "          const element = document.querySelector('#df-4cb56634-426a-4b4c-b762-54da095d3924');\n",
       "          const dataTable =\n",
       "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                     [key], {});\n",
       "          if (!dataTable) return;\n",
       "\n",
       "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "            + ' to learn more about interactive tables.';\n",
       "          element.innerHTML = '';\n",
       "          dataTable['output_type'] = 'display_data';\n",
       "          await google.colab.output.renderOutput(dataTable, element);\n",
       "          const docLink = document.createElement('div');\n",
       "          docLink.innerHTML = docLinkHtml;\n",
       "          element.appendChild(docLink);\n",
       "        }\n",
       "      </script>\n",
       "    </div>\n",
       "  </div>\n",
       "  "
      ]
     },
     "metadata": {},
     "execution_count": 35
    }
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "##8.Evaluate Predictions "
   ],
   "metadata": {
    "id": "y03ZPigmGPYL",
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "from sklearn.metrics import classification_report\n",
    "print(classification_report(predctions['y'], predctions['questionpair']) )"
   ],
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "yfJtQLGDu8nN",
    "outputId": "4da6ed4b-dbc4-4855-b9dd-fcdbb0db79cd",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      " almost_same       0.60      0.86      0.71      5590\n",
      "    not_same       0.89      0.66      0.75      9410\n",
      "\n",
      "    accuracy                           0.73     15000\n",
      "   macro avg       0.74      0.76      0.73     15000\n",
      "weighted avg       0.78      0.73      0.74     15000\n",
      "\n"
     ]
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VXu21c0iQRSC",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# There are many more models you can put to use in 1 line of code!\n",
    "## Checkout [the Modelshub](https://nlp.johnsnowlabs.com/models) and the [NLU Namespace](https://nlu.johnsnowlabs.com/docs/en/spellbook) for more models\n",
    "\n",
    "### NLU Webinars and Video Tutorials\n",
    "- [NLU & Streamlit Tutorial](https://vimeo.com/579508034#)\n",
    "- [Crash course of the 50 + Medical Domains and the 200+ Healtchare models in NLU](https://www.youtube.com/watch?v=gGDsZXt1SF8)\n",
    "- [Multi Lingual NLU Webinar - Tutorial on Chinese News dataset](https://www.youtube.com/watch?v=ftAOqJuxnV4)\n",
    "- [John Snow Labs NLU: Become a Data Science Superhero with One Line of Python code](https://events.johnsnowlabs.com/john-snow-labs-nlu-become-a-data-science-superhero-with-one-line-of-python-code?hsCtaTracking=c659363c-2188-4c86-945f-5cfb7b42fcfc%7C8b2b188b-92a3-48ba-ad7e-073b384425b0)\n",
    "- [Python Web Def Conf - Python's NLU library: 1,000+ Models, 200+ Languages, State of the Art Accuracy, 1 Line of Code](https://2021.pythonwebconf.com/presentations/john-snow-labs-nlu-the-simplicity-of-python-the-power-of-spark-nlp)\n",
    "- [NYC/DC NLP Meetup with NLU](https://youtu.be/hJR9m3NYnwk?t=2155)\n",
    "\n",
    "### More ressources \n",
    "- [Join our Slack](https://join.slack.com/t/spark-nlp/shared_invite/zt-lutct9gm-kuUazcyFKhuGY3_0AMkxqA)\n",
    "- [NLU Website](https://nlu.johnsnowlabs.com/)\n",
    "- [NLU Github](https://github.com/JohnSnowLabs/nlu)\n",
    "- [Many more NLU example tutorials](https://github.com/JohnSnowLabs/nlu/tree/master/examples)\n",
    "- [Overview of every powerful nlu 1-liner](https://nlu.johnsnowlabs.com/docs/en/examples)\n",
    "- [Checkout the Modelshub for an overview of all models](https://nlp.johnsnowlabs.com/models) \n",
    "- [Checkout the NLU Namespace where you can find every model as a tabel](https://nlu.johnsnowlabs.com/docs/en/spellbook)\n",
    "- [Intro to NLU article](https://medium.com/spark-nlp/1-line-of-code-350-nlp-models-with-john-snow-labs-nlu-in-python-2f1c55bba619)\n",
    "- [Indepth and easy Sentence Similarity Tutorial, with StackOverflow Questions using BERTology embeddings](https://medium.com/spark-nlp/easy-sentence-similarity-with-bert-sentence-embeddings-using-john-snow-labs-nlu-ea078deb6ebf)\n",
    "- [1 line of Python code for BERT, ALBERT, ELMO, ELECTRA, XLNET, GLOVE, Part of Speech with NLU and t-SNE](https://medium.com/spark-nlp/1-line-of-code-for-bert-albert-elmo-electra-xlnet-glove-part-of-speech-with-nlu-and-t-sne-9ebcd5379cd)"
   ]
  }
 ]
}